Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Support filtering the /messages API by relation type (MSC3874) #14148

Merged
merged 2 commits into from
Oct 17, 2022

Conversation

clokep
Copy link
Member

@clokep clokep commented Oct 11, 2022

Provides an experimental config option to enable support for MSC3874, filtering of /messages by the relation type of events.

Related to #14143.

@clokep clokep force-pushed the clokep/msg-filtering branch from 784c823 to 7485128 Compare October 14, 2022 19:31
@clokep clokep changed the title Support MSC3874. Support filtering the /messages API by relation type (MSC3874) Oct 14, 2022
@clokep clokep marked this pull request as ready for review October 14, 2022 19:36
@clokep clokep requested a review from a team as a code owner October 14, 2022 19:36
@clokep clokep force-pushed the clokep/msg-filtering branch from 5d3ce2b to e2e514e Compare October 14, 2022 19:51
"event_relation.relation_type != ?" for _ in event_filter.not_rel_types
)
)
args.extend(event_filter.not_rel_types)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine for now, but I am concerned about the query performance of these fields. In particular, I think for uncommon relation types this could cause us to scan the full events table looking for matches?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you concerned specifically for the not_rel_types case or for both cases?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran a couple of queries against Matrix HQ, one for not m.thread and one for not invalid-rel-type. Both have similar (identical?) query plans:

`m.thread` case
=>EXPLAIN ANALYZE SELECT
->     event.event_id, event.instance_name,
->     event.topological_ordering, event.stream_ordering
-> FROM events AS event
-> LEFT JOIN event_relations AS event_relation USING (event_id)
-> WHERE
->     event.outlier = false
->     AND event.room_id = '!OGEhHVWSdvArJzumhm:matrix.org'
->     AND ((505473,3359746309) < (event.topological_ordering,event.stream_ordering))
->     AND ((event_relation.relation_type != 'm.thread') OR event_relation.relation_type IS NULL)
-> ORDER BY event.topological_ordering DESC,
-> event.stream_ordering DESC LIMIT 20;
                                                                             QUERY PLAN                                                                              
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=1.27..134.20 rows=20 width=73) (actual time=0.775..6.941 rows=20 loops=1)
   ->  Nested Loop Left Join  (cost=1.27..154360.09 rows=23224 width=73) (actual time=0.774..6.937 rows=20 loops=1)
         Filter: ((event_relation.relation_type <> 'm.thread'::text) OR (event_relation.relation_type IS NULL))
         ->  Index Scan Backward using events_order_room on events event  (cost=0.70..48115.51 rows=23288 width=73) (actual time=0.304..0.500 rows=20 loops=1)
               Index Cond: ((room_id = '!OGEhHVWSdvArJzumhm:matrix.org'::text) AND (ROW(505473, '3359746309'::bigint) < ROW(topological_ordering, stream_ordering)))
               Filter: (NOT outlier)
         ->  Index Scan using event_relations_id on event_relations event_relation  (cost=0.57..4.55 rows=1 width=55) (actual time=0.320..0.320 rows=0 loops=20)
               Index Cond: (event.event_id = event_id)
 Planning time: 0.281 ms
 Execution time: 6.974 ms
(10 rows)
`invalid-rel-case` case
=> EXPLAIN ANALYZE SELECT
    event.event_id, event.instance_name,
    event.topological_ordering, event.stream_ordering
FROM events AS event
LEFT JOIN event_relations AS event_relation USING (event_id)
WHERE
    event.outlier = false
    AND event.room_id = '!OGEhHVWSdvArJzumhm:matrix.org'
    AND ((505473,3359746309) < (event.topological_ordering,event.stream_ordering))
    AND ((event_relation.relation_type != 'invalid-rel-type') OR event_relation.relation_type IS NULL)
ORDER BY event.topological_ordering DESC,
event.stream_ordering DESC LIMIT 20;
                                                                             QUERY PLAN                                                                              
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=1.27..134.20 rows=20 width=73) (actual time=0.069..0.362 rows=20 loops=1)
   ->  Nested Loop Left Join  (cost=1.27..154360.09 rows=23224 width=73) (actual time=0.068..0.356 rows=20 loops=1)
         Filter: ((event_relation.relation_type <> 'invalid-rel-type'::text) OR (event_relation.relation_type IS NULL))
         ->  Index Scan Backward using events_order_room on events event  (cost=0.70..48115.51 rows=23288 width=73) (actual time=0.050..0.105 rows=20 loops=1)
               Index Cond: ((room_id = '!OGEhHVWSdvArJzumhm:matrix.org'::text) AND (ROW(505473, '3359746309'::bigint) < ROW(topological_ordering, stream_ordering)))
               Filter: (NOT outlier)
         ->  Index Scan using event_relations_id on event_relations event_relation  (cost=0.57..4.55 rows=1 width=55) (actual time=0.011..0.011 rows=0 loops=20)
               Index Cond: (event.event_id = event_id)
 Planning time: 0.652 ms
 Execution time: 0.425 ms
(10 rows)

@clokep clokep merged commit 4283bd1 into develop Oct 17, 2022
@clokep clokep deleted the clokep/msg-filtering branch October 17, 2022 15:32
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Oct 29, 2022
Upstream changes:

Synapse 1.70.1 (2022-10-28)
===========================

(bugfixes)


Synapse 1.70.0 (2022-10-26)
===========================

Features
--------

- Support for
  [MSC3856](matrix-org/matrix-spec-proposals#3856):
  threads list
  API. ([\#13394](matrix-org/synapse#13394),
  [\#14171](matrix-org/synapse#14171),
  [\#14175](matrix-org/synapse#14175))

- Support for thread-specific notifications & receipts
  ([MSC3771](matrix-org/matrix-spec-proposals#3771)
  and
  [MSC3773](matrix-org/matrix-spec-proposals#3773)). ([\#13776](matrix-org/synapse#13776),
  [\#13824](matrix-org/synapse#13824),
  [\#13877](matrix-org/synapse#13877),
  [\#13878](matrix-org/synapse#13878),
  [\#14050](matrix-org/synapse#14050),
  [\#14140](matrix-org/synapse#14140),
  [\#14159](matrix-org/synapse#14159),
  [\#14163](matrix-org/synapse#14163),
  [\#14174](matrix-org/synapse#14174),
  [\#14222](matrix-org/synapse#14222))

- Stop fetching missing `prev_events` after we already know their
  signature is
  invalid. ([\#13816](matrix-org/synapse#13816))

- Send application service access tokens as a header (and query
  parameter). Implements
  [MSC2832](matrix-org/matrix-spec-proposals#2832). ([\#13996](matrix-org/synapse#13996))

- Ignore server ACL changes when generating pushes. Implements
  [MSC3786](matrix-org/matrix-spec-proposals#3786). ([\#13997](matrix-org/synapse#13997))

- Experimental support for redirecting to an implementation of a
  [MSC3886](matrix-org/matrix-spec-proposals#3886)
  HTTP rendezvous
  service. ([\#14018](matrix-org/synapse#14018))

- The `/relations` endpoint can now be used on
  workers. ([\#14028](matrix-org/synapse#14028))

- Advertise support for Matrix 1.3 and 1.4 on
  `/_matrix/client/versions`. ([\#14032](matrix-org/synapse#14032),
  [\#14184](matrix-org/synapse#14184))

- Improve validation of request bodies for the [Device
  Management](https://spec.matrix.org/v1.4/client-server-api/#device-management)
  and [MSC2697 Device
  Dehyrdation](matrix-org/matrix-spec-proposals#2697)
  client-server API
  endpoints. ([\#14054](matrix-org/synapse#14054))

- Experimental support for
  [MSC3874](matrix-org/matrix-spec-proposals#3874):
  Filtering threads from the `/messages`
  endpoint. ([\#14148](matrix-org/synapse#14148))

- Improve the validation of the following PUT endpoints:
  [`/directory/room/{roomAlias}`](https://spec.matrix.org/v1.4/client-server-api/#put_matrixclientv3directoryroomroomalias),
  [`/directory/list/room/{roomId}`](https://spec.matrix.org/v1.4/client-server-api/#put_matrixclientv3directorylistroomroomid)
  and
  [`/directory/list/appservice/{networkId}/{roomId}`](https://spec.matrix.org/v1.4/application-service-api/#put_matrixclientv3directorylistappservicenetworkidroomid). ([\#14179](matrix-org/synapse#14179))


Deprecations and Removals
-------------------------

- Remove the experimental implementation of
  [MSC3772](matrix-org/matrix-spec-proposals#3772). ([\#14094](matrix-org/synapse#14094))

- Remove the unstable identifier for
  [MSC3715](matrix-org/matrix-spec-proposals#3715). ([\#14106](matrix-org/synapse#14106),
  [\#14146](matrix-org/synapse#14146))
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants